(1) Field of the Invention
The present invention relates to an image processing apparatus and method that is capable of extracting a desired object from a sequence of images of a motion picture on the basis of a detected contour of the object while requiring only a simple operation from the operator.
Recently, low-cost, high-performance personal computers and memories have come into widespread use, which enables easy manipulation of image data of a color image or a motion picture by the operator of the personal computer. To create a motion picture that is viewed by other users on the personal computers for advertisement uses, educational uses, or entertainment uses, there is an increasing demand for a tool which is capable of editing images of the motion picture while only requiring a simple operation from the operator.
Editing the images of a motion picture involves several image processing processes, such as object-contour detection, object extraction, color-changing, enlargement, reduction, transfiguration, composition and/or recognition. The first process that is performed at the beginning of these image processing processes is a detection of a contour of a desired object (for example, an automobile or a human being) from image frames of a motion picture. In particular, the present invention relates to such an image processing apparatus and method which is capable of extracting a desired object from a sequence of image frames of a motion picture on the basis of a detected contour of the object.
(2) Description of the Related Art
U.S. Pat. No. 5,475,507 (corresponding to Japanese Laid-Open Patent Application No.6-203157), which is assigned to the assignee of the present invention, discloses a color image processing method which detects a contour of a desired object in a color image containing only stationary components.
In the method of the above-mentioned publication, when the operator has designated a point at a position near the contour of the desired object in the image being processed, a boundary point on the contour of the object is detected on the basis of the designation. From the detected boundary point, other points on the object-contour are subsequently detected to detect the object-contour.
However, the method of the above-mentioned publication is intended to detect the object-contour from the image containing only stationary components. If this method is applied to the detection of the object-contour in a sequence of image frames of a motion picture, the operator must designate a point at a position near the contour of the object in a related image being processed for every image frame of the motion picture. Such an image processing method requires unnecessary redundant operations from the operator, and the operability is considerably low.
Further, Japanese Laid-Open Patent Application No.7-320068 (corresponding to Japanese Patent Application No.6-109914) discloses an image processing method for detecting an edge of an outline of a desired object from a sequence of image frames of a motion picture and detecting the object-contour on the basis of the detected edge.
In the method of the above-mentioned publication, an average of density levels of neighboring picture elements at an edge of the object-contour of a preceding image frame is detected. A threshold level which is used to detect a corresponding edge of the object-contour of a following image frame is determined by the average of the density levels for the preceding image frame.
The object-contour detection method of the above-mentioned publication involves two basic methods: one utilizing a motion of the object to separate the object-contour from the background region in the image frames (which is called a motion-based contour detection), and the other utilizing a color of the object to detect the object-contour in the image frames (which is called a color-based contour detection).
In the motion-based contour detection of the above-mentioned publication, a reference image containing only stationary components is compared with a subsequent image having the same environment but including a moving object. This comparison results in the difference of the two images canceling the stationary components, leaving only nonzero entries that correspond to the nonstationary image components. However, the essential requirement of this method is that there is an appreciable difference in the two images with respect to the moving object included therein. If there is no difference in the two images with respect to the moving object, this method cannot detect the contour of the desired object and the reliability of the detection is low. Further, if there is another moving object different than the desired moving object in the motion picture, this method may detect the contours of both the moving objects, and the reliability of the detection is low.
On the other hand, in the color-based contour detection of the above-mentioned publication, only color values of pixels of the desired object are detected to detect the object-contour in the image frames. However, the essential requirement of this method is that there is an appreciable difference between the color of the object and the color of the background. If there is no appreciable color difference of the object and the background, this method cannot detect the contour of the object of interest and the reliability of the detection is low.